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(57) Abstract: The invention provides a method of processing messages. The method includes the step of obtaining a subscriber 
message, from a subscriber communication device over a telecommunications network, the subscriber message representing a voice 
message from a subscriber. The subscriber message is stored in computer memory and then presented in an audio form to a human 
operator. A data file is composed which is derived from the subscriber message and the data file transmitted to a subscriber computing 
device and/or the subscriber conununication device. The invention provides a related system for processing messages. 
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METHOD AND SYSTEM OF PROCESSING MESSAGES 
FIELD OF INVENTION 

5 The invention relates to method(s) and process(s) i:ised for transmitting data between 
electronic messaging systems, independent of the voice message form that the data was 
originated, and more particularly to provide an efficient means of allowing users to update 
data stored on a personal computer using a communications device, such as a mobile device 
connected to cellular network or telephone connected to the public switched telq)hone 

10 network (PSTN). 

BACKGROUND TO INVENTION 

15 The use of personal computing devices by consumers is growing. The number of personal 
confuting devices has grown substantially in both the type and munber. To cope with 
increasingly busy modem lifestyles, individuals are increasingly using personal computing 
devices running sophisticated e-mail and personal infomoation manager (PIM) conyuter 
programs, to assist them organise their lives. These computer programs run on personal 

20 computers in fixed locations and on mobile devices. 

Mobile devices include cellular phones that now have enough memory and processing 
power to store contact information, text message, surf the Web, and picture/video 
messaging. Personal computers are no longer relegated to just desktop or notebook 
25 computers, increasingly they are being miniaturized to more closely resemble traditional 
mobile devices. For example, small hand-held devices such as personal digital assistants 
(PDA) have evolved into powerful devices, such as Palm Pilot and Pocket PC, and these 
' are now being integrated with mobile phones to form hybrid mobile phone/pocket PC 
mobile devices. 

30 
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Many users, in their enthusiasm to embrace new technology, find it difBcult and time 
consimiing to enter information on their different personal conq>uting devices For 
example, many of today's cell phones have a diary feature which users may not use often 
due to the number of steps necessary to enter appointment information. To overcome this 
5 problem, some mobile devices have xmique interfaces such as writing pads or miniature 
keyboards to facilitate recording information and updating calendars, phone Usts and other 
features of mobile devices. Many users also use desk top or PC based organizer software 
such as Microsoft's personal con5)uter-based "Outlook" and regularly use other software 
in tiieir day to day activities but are unable to always have access to that desk top computer 
10 at all times. 

In response to these problems, many users make use of synchronization computer programs 
or network-based services to periodically replicate data between disparate information 
stores. One exan5)le of this involves transferring data directly between Microsoft's 

15 personal con5)uter-based "Outlook" and pocket PC-based "Pocket Outlool^' e-mail 
applications via direct cable/wireless local area connection between ttie devices. Other 
systems maintain a single personal calendar data store and neglect the rest For example, 
users may keep their e-mail program's personal calendar updated but not their mobile 
device's personal calendar program. For senior executives or business owners, often the 

20 later approach is used in conjunction with the services of a secretary or personal assistant, 
with the appropriate rights and permissions to access and make changes to the user's e-mail 
program's information and data. 

One solution is described in WO 01/74023 to Sun Microsystems Inc. It describes a system 
25 in which a mobile device is registered with a server and tbe user of that mobile device 
transmits coramands to the server in a particular data format for remotely managing and/or 
• accessing information stored on a network device. Such a data format can often be difficult 
and time-consuming for a consumer to adopt. 
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SUMMARY OF mVENTION 

In one fonn the invention comprises a method of processing messages comprising the steps 
of obtaining a subscriber message, from a subscriber communication device ov» a 
5 telecommunications network, the subscriber message representing a voice message from a 
subscriber; storing the subscriber message in computer memory; presenting the subscriber 
message in an audio form to a hxmian operator; composing a data file derived from the 
subscriber message; and transmitting the data file to a subscriber computing device and/or 
the subscriber communication device. 

10 

In another fomi the invention comprises a system for processing messages comprising a 
mail system configured to obtain a subscriber message from a subscriber communication 
device over a telecommunications network, the subscriber device representing a voice 
message from a subscriber; a database maintained in computer memory configured to store 
15 the subscriber message; a retrieval component configured to retrieve a subscriber message 
from the database and to present the subscriber message in an audio fomi to a human 
operator; and a data entry component configured to obtain data from the human operator 
derived from the subscriber message. 

20 BRIEF DESCRIPTION OF THE FIGURES 

Preferred forms of the method and system of processing messages will now be described 
with reference to the accompanying figures in which: 

25 Figure 1 shows a block diagram of a system in which forms of the inv^tion may be 
inq>lemented; 

Figure 2 illustrates a preferred form initial screen presented to an operator; 
30 Figure 3 shows a preferred form contact entry form; 
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Figure 4 shows a preferred form appointment entry form; 
Figure 5 shows a preferred form task entry fonn; and 

5 

Figure 6 shows a preferred form data entry form for imintelligible messages. 
DETAILED DESCRIPTION OF PREFERRED FORMS 

10 Figure 1 illustrates a block diagram of the preferred system 10 in which forms of the 
present invention may be in5>lemented. The invention provides a method and system for a 
subscriber to a service, for example subscriber 20 or subscriber 30, to transmit data in the 
form of either (1) a voice call from telephone 40 or mobile device 50 or (2) a mobile 
originated text message (MO e-mail) using mobile device 50 to a sxibscriber conoputing 

15 device, for example computing device 60 or coniputmg device 70. 

Mobile devices 40 and 50 could include for example a cellular phone, a two-way pager, or 
mobile computer. Computing devices 60 and 70 could include for example a personal 
PDA, a personal computer, or other computer system running the appropriate application 
20 software. Such a computer system could be operated by either subscriber 20 or 30 or a 
trusted third party, such as a hospitality service provider or travel agent. The computing 
devices 60 and 70 could also include purpose-built computing devices installed for example 
at a subscriber's premises. 

25 To initiate an operator-assisted process, the subscriber 20 or 30 originates eitfier (1) a voice 
call to a voice mail or Internet protocol (IP voicemail) gateway using a mobile device 40 or 
• 50, or (2) a mobile origmated text message using the mobile device. The inbound 
subscriber voice call to the service or mobile originated text message is then transmitted 
over a telecommunications, PSTN or mobile network. The identity of the subscriber 

30 initiating an operator-assisted process can be identified by either (1) the calling party 



SUBSTITUTE SHEET (Rule 26) 



wo 03/051026 



PCT/NZ02/00275 



5 

number (A number) for voice calls and also the rqjly email address, in tihie case of mobile 
originated e-mail messages. 

The voice call or mobile originated text message is passed throu^ the telecommimication 
5 network 80 to a cont^juter host or mail system 90. The system 90 includes call termination 
equipment which turns the call into an H.323 data stream and passes the data stream to a 
web server 100 which in one form could be conjBgured as an H.323 gatekeeper server. Hie 
gatekeeper server passes the H.323 data stream to an H.323 voice recording server. It will 
be appreciated that the gatekeeper server and the voice recording server could be 
10 iinplemented on the same or different conq)uter equipment. 

The mail system 90 includes a database 110 maintained in computer memory and a 
processor 120. The database 110 could have stored in it known subscribers to the service. 
The database could be stored as a relational database with each record of the database 
15 representing a different subscriber and each record including or identified by the calling 
party number. 

In the case of a voice call, tlie voice recording server extracts the calling party number fix>m 
the voice call and checks database 1 10 to determine whether or not the voice call belongs to 
20 a subscriber of this service. 

If the calling party munber of the voice call matches a calling party number in the database 
110, then the voice recording server assumes that the subscriber who has sent the voice call 
is a subscriber to the service. The server plays a message to the subscriber prompting the 
25 subscriber to leave a message. 

' On the other hand, if the calling party number does not match a calling party number stored 
in the database 110, the voice recording server assumes that the subscriber is not a 
subscriber to the service and returns a message to the subscriber at the calling party number 
30 saying that the subscriber is not registered for flie service and to contact customer services. 
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The voice recording server translates the voice call firom fhe subscriber to an audio file. 
The H.323 server creates a new record in fhe database 110 containing fhe audio file 
togeflier with metadata, for example the date, time and the calling party number relevant to 
5 the audio file. 

It is envisaged that fhe message could first be stored on mobile device 40 or 50 and 
transmitted over telecommunications network 80 at a later date. This buffering function 
would be useful, for example, where a particular mobile device is outside of a coverage 
10 area and unable to transmit a voice call or text message iromediately. 

Each inbound voice call or text message sent by a subscriber is received by a voice mail, or 
an IP voicemail server, or an e-mail system (collectively also known as a messaging 
system) located either on (1) fhe message-processing service bureau's premises or on the 
15 telecommunications network 80. All inbound messages are stored in eith^ (1) a 
computer-based message store (MS) 110 for retrieval by a single operator (also known as 
an agent) or (2) a network-based post office (PO) or message store server (MSS) 110 for 
distribution and delivery to the e-mail client of two or more operators (agents). Exan^>les 
of the stored subscriber messages are indicated at 130. 

20 

It is envisaged that the subscriber messages be stored in computer memory in an 
appropriate form. For example, (1) where the subscriber voice calls are terminated by an 
enterprise-based voice mail gateway that supports Microsoft's telephony {^plication 
programming interface (TAPI), then voice messages would be converted and stored as e- 
25 mail messages (ASCII and binary); (2) where subscriber voice calls are temunated by a 
network-based IP voicemail server then voice messages would undergo Intemet protocol 
' conversions, to support TCP/IP transport and multipmpose Intemet mail extension (MIME) 
e-mail format, before being delivered by a siniple message transfer protocol (SMTP) to the 
message-processing service bureau's e-mail system; and (3) where subscriber MO e-mail is 
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delivered by a network-based message transfer agent (MTA) using SMTP, then the 
message would be stored as an ASCII text message. 

The above message collection and storage processes, depending on the desired scalability 
5 of the service, would all allow voice call originated messages to eventually be collected, 
stored, and distributed to other parts of the system as email messages, including ASCII text 
and binary siq)port, for exanq)le with .wav (audio) ffle attachments. 

In one preferred form of ttie iuvention, each message stored in computer memory is 
10 allocated a message identifier (message ID) and a subscriber identifier (user ID) where the 
subscriber is identified fiom the mobile device's A number from which the message 
originated. The time and date that the message was transmitted from the mobile device or 
the time and date the message was received at the mail system could form part of the 
message store server (MSS) database in the messaging system computer memory. In this 
15 way, the messages 130 could be stored in an ordered sequence, for example a first in first 
out (FIFO) queue. It is envisaged that other ordered sequences coidd be inq>lemented, for 
example last in first out (LBFO) or alternatively individual subscribers could be ordered in 
the sequence ahead of oth^ subscribers. 

20 Each subscriber message is fetched from database 1 10 by processor 120 and transmitted as 
an audio file 140 over communications network 150 or otherwise to an operator, for 
example operator 160, operator 170, or operator 180 in a voice message form, i.e. an e-mail 
with .wav audio attachment, as an output from an e-mail program running on multimedia 
coiiq)uter equipped with a soimd card and audio headset or speakers- Where more than one 

25 subscriber message is stored in database 110, it is envisaged that queued subscriber 
messages be distributed from the con^uter memory to an operator who is available for 
* processing messages (ie logged in) and who is not currenfly processing a message. Where 
the subscriber messages form an ordered sequence, it is envisaged that successive 
subscriber messages be retrieved from the corr5>uter memory 100 in accordance with the 

30 location in the message queue. 
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It is envisaged that each operator be associated with or at least have access to respective 
workstations 190, 200 or 210, The workstations preferably include a con5)uter memory, a 
processor, a display device and an audio device. 

5 

The audio file 140 is passed to the workstation of the next available operator. Software 
installed on the workstation, which is described below, causes the audio file to be played 
through flie audio device to the operator. The operator listens to the message and actions 
the message based on the content of the message. Using software installed on the 
10 workstation, the operator processes the message. On conq>letion, a data file 200 is 
transmitted fit)m the operator workstation over the communications network 150 back to 
the mail system 90. 

The mail system 90 then sends data file 210 through communications network 150 to 
15 computing device 60 or 70 in a format pre-selected by the subscriber 20 or 30. It is also 
envisaged tiiat the mail system 90 transmits a confinnatory message 220 over the 
telecommunication network or mobile network 80 to subscriber device 40 or 50. 

It is envisaged that subscribers 20 and 30 pre-register with tihte system. The registration 
20 process would specify the communication networks and the con:q>uting devices available to 
the system over which to transmit operator messages. 

The operator could, for exanq^le, receive a request from a subscriber to enter a contact into 
Microsoft Outlook. The operator receives the verbal instruction in the form of a voice 
25 message firom the subscriber and enters the requested data into an operator workstatiorL 
The data would then be transmitted over a communicatioiis network in the form of a local 
* area network or LAN, Internet or other suitable network in the form of a self-extracting file 
or equivalent to a computing device ra the form of a subscribing workstation on which 
Microsoft Outlook is installed and operating. It is also envisaged that the subscriber be 
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registered to send other data items, for exa]iq}le calendar entries, alarms notes or tasks to 
subscriber device 40 or 50. 

It is also envisaged that a subscriber may pre-register to enable an operator to initiate, 
5 process or transmit requests to a third party person(s) or a third party coniputing device in 
the form of a restaurant booking system or a travel agent booking system or to a user 
accoimting system 

It is also envisaged that a subscriber may pre-register to enable an operator to transmit 
10 requests to a mobile phone in the form of a self extracting mobile telephony instructions for 
adding names and contacts, setting alarms and utiLisiag other functions of the mobile 
phone. 

As described above, the system could transmit a confirmatory message 220 over 
15 telecommunications network 80 back to the originating mobile device, as is shown with 
mobile device 50. Such a confirmatory message in the form of a text message could be 
transmitted following successfiil transmission of the operator message to the subscriber 
computing device over the communications network. 

20 In a fiuiher preferred form of the invention the mail sj^tem 90 could include, or at least be 
interfaced to, a device for preserving the anonymity of a subscriber. Such a device could, 
for exanq>le, alter the pitch or tone of a subscriber message when presented to an operator 
or could slow down or speed up such a message to ensure that the operator is not able to 
identify the subscriber. 

25 

In a further preferred form of the invmtion the subscriber 20 or 30 may choose to have a 
* key word or code word which will form part of the subscriber message enabling the system 
to verify the identification of subscriber 20 or 30 and thereby making available secure 
services such as making purchases firom a retailer on behalf of the subscriber. 

30 
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It is also envisaged that a siibscriber may pre-registCT to enable an operator to transmit 
command strings to a co£tq)uting device in die form of standard int^face protocols such as 
RS232 in the case of generic devices or proprietary communications commands for 
example to a home automation system and/ or "smart appliances" owned by a subscriber. 
5 Such a device could for example enable an operator to transmit requests to a conqputiag 
device controlling a cable or digital broadcast system to schedule or receive transmissions. 
In this case confuting device(s) 60 or 70 could include a home automation control system 
and/or smart appliances. 

10 The software installed on operator workstations is now described in more detail. The 
operator preferably uses a web browser to enter data into fomos on the server ftom which 
data fiOies are transmitted to system 90. Figure 2 illustrates an initial screen or form which 
could be presented to an operator indicating that a new message is in die queue. The audio 
file representing the message is played to the operator using the audio device on the 

15 operator workstation. The message continually plays until the operator has finished 
processing the message. A control bar 310 could be provided which enables an operator to 
stop or pause a particular message, or to skip to the previous message, the next message or 
the first or last message in a queue. It is envisaged that the skip fimction could be used if 
the message should be passed to a supervisor or is perhaps noise requiring no action. 

20 

Generally each subscriber message will represent a contact, an appointment or a task or a 
subscriber specific task such as a booking request or home automation command request 
The operator decides whether a message fits best within a contact, appointment or task and 
selects the appropriate element by clicking on contact button 320, appointment button 330 
25 or task button 340 respectively which are further described below. If tBe message is 
unintelligible, the operator selects the unintelligible button 350 or can elect to skip the 
' message by selecting skip button 360. 

Figure 3 illustrates a contact entry form. The operator is able to enter data such as name, 
30 title, organization, address, mobile phone, business phone, fax, home phone, other phone 
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and/or email. A "contact" message represents a new contact the subscriber wishes to be 
sent to the co^^>uting device 60 or 70. 

Preferably the web form provides hot keys for the operator taking the data entry cursor 
5 directly to a relevant field. For exancqple, Alt-1 would move flie data entry cursor to the 
name field and Alt-S would take the data entry cursor to the mobile phone number. 

It is possible that the same message may contain more tiian one element For exan9>le, the 
message may contain one or more different contacts, ^pointments or tasks. If the message 
10 contains a fiirther element, the operator can enter a new contact, appointment or task by 
selecting contact, gqppointment or task buttons 320, 330 and 340. If the next element is 
unintelligible, flie operator selects the unintelligible button 350. Otherwise, the user clicks 
the s&ad button 370 to compile the data into a data file and transmit the data file to the mail 
system 90. 

15 

If the audio file contains a new appointment and the operator has recognized this part of the 
audio file as representing an appointment, the user selects the appointment user entry form 
and an exanq)le form is shown in Figure 4. The user could enter, for example simimary, 
description, location, time, date, duration and alarm, depending on the requirements of the 
20 user. Once again, hot keys could be provided for the operator to skip between different 
fields. On conq>letion of the data entry fields, the opemtor selects the send button 370 to 
transmit the data, or brings xsp fiirther contact, appointment or task user entry fomis. 

Figure 5 illustrates a preferred data entry form for entering a task in which the operator is 
25 provided with a summary data entry field, a description field and a due date. ""'^ 

* Hot keys could be provided to the operator to switch quickly between different fields. On 
completion, the user selects the send button 370 or one of the otiier buttons depending on 
the content of the message. 

30 
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In some circumstances, the subscribe message may be partly or wholly unintelligible. For 
exan^jle, the message could clearly relate to a contact but the phone number for this contact 
is not clear. Altematively, the message could include or represent an appointment but the 
time is not audible. In these circumstances, the operator selects the uninteUigible button 
5 bringing up the xmintelligible data entry form shown in Figure 6. There is preferably only 
one field, a context field which allows the operator to enter data in the form of a text 
explanation stating in what context the message was not intelligible and a text message and 
email could be sent to the subscriber 20 or 30 indicating that the subscriber will need to 
retransmit or create a new subscriber message. 

10 

hi one form, the invention provides a method to enable a subscriber to add contacts, 
q)pointments and tasks to computer organizer software and/or a cellphone. 

To add a contact for exan:q>le, a subscriber dials the appropriate messaging number and 
15 waits for a prompt. After the pron5)t, the subscriber states **new contact" or similar, 
provides name, conq)any, phone, email and/or physical address details and hangs up. A 
minute or so later an ^ail arrives on the subscriber's con9>uter with the new contact in it. 
The subscriber opens the email, making any necessary additions or changes, and saves the 
file in the subscriber's list of contacts. A confirmatory message can also be sent to the 
20 subscriber's cellphone indicating a new contact and prompting the subscriber to viQW 
and/or save the contact in the address book of the cellphone. 

To add a new appointment, a subscriber dials tiie same messaging number and waits for the 
prompt. After the prompt, the subscriber states **new appointment" and provides details. A 

25 minute or so later, an email arrives on the subscriber's computer with the new appointment 
in it. The subscriber is able to open the email, making any necessary additions or changes, 
• and saves the new appointment in the subscriber's diary. A confirmatory message can also 
be sent to the user's cellphone showing a new calendar note received and prompting the 
user to view and/or save the new appointment in the calendar of the phone. The default is 

30 to insert an alarm in the calendar entry so that the subscriber is reminded about the 
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^pointment by the phone making an alann sound or a pop-up dialogue box on the 
subscriber's conqiuter. 

To add a task for example, the subscriber dials tho same messaging number and waits for 
5 the prompt After the prompt, the subscriber states '^ew task" and provides details. A 
minute or so later an email arrives on the subscriber's conq>uter with the new task in it 
The subscriber opens the task and adds the new task to the subscriber's task Ust. Where the 
user's cellphone siq}ports tasks, a confirmatory message could be s^t to the user's 
cellphone which beeps for attention and shows the new task received, pronq>ting the user to 
1 0 view and/or save the task in the task list on the plhone. 

The subscriber can place further instructions in each message. For example, the subscriber 
could send the message **new contact, Reading Cinemas, Courtney Place, Wellington — 
look up the phone number please" and the subscriber receives the contact with the phone 
1 5 number inserted in the contact as an alternative to calling directory assistance. 

The subscriber may further place specific **preset" instructions in each message. For 
example, the subscriber coiild send the message ^Vake up for tomorrow 6 am TV" and die 
subscriber's home automation sj^em receives the appropriate command string to initiate a 
20 preset wake up mode and alarm time on that system for the given day or time. This could in 
this example open the blinds in the subscribers bedroono, turn on a preset Ughting "scene" 
or start some favorite music or TV programme in a particular zone of the system at the 
given time. 

25 The foregoing describes the invention including preferred forms thereof, Afterations and 
modifications as will be obvious to those skilled in the art are intended to be incorporated 
' within the scope hereof, as defined by the acconq>anying claintis. 
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CLAIMS: 

1 . A method of processing messages comprising tiie steps of: 

5 obtaining a subscriber message, from a subscriber communication device over a 

telecommunications network, the subscriber message representing a voice message from a 
subscriber; 

storing the subscriber message in computer memory; 
presenting the subscriber message in an audio form to a human operator; 
10 conq)osing a data file derived from the subscriber message; and 

transmitting the data file to a subscriber computing device and/or the subscriber 
communication device. 

2. A method of processing messages as claimed in claim 1 fiirther comprising the steps 
15 of storing two or more subscriber messages in conq>uter memory; and retrieving successive 

subscriber messages from computer memory to present to the human operator. 

3. A method of processing messages as claimed in claim 1 or claim 2 fiirfher 
conq}rising the step of linking the subscriber message with a unique identifier code and 

20 storing the subscriber message with the unique identifier code. 

4. A method of processing messages as claimed in any one of the preceding claims 
fiirther conq)rising the step of transmitting the subscriber message in addition to the 
operator message to the subscriber computing device or communications device. 

25 

5. A method of processing messages as claimed in any one of the preceding claims 
' fiirther comprising tibe steps of queuing and retrieving successive subscriber messages from 

computer memory in an ordered sequence. 



SUBSTITUTE SHEET (Rule 26) 



wo 03/051026 



PCT/NZ02/00275 



15 

6. A method of processing messages as claimed in any one of the preceding claims 
further comprising the step of distributing and presenting Hie subscriber message to one of a 
plurality of human operators. 

5 7. A system for processing messages corqprising: 

a mail system configured to obtaiti a subscriber message firom a subscriber 
communication device over a telecommunications network, the subscriber device 
representing a voice message firom a subscriber; 

a database maintained in computer memory configured to store the subscriber 
10 message; 

a retrieval component configured to retrieve a subscriber message from the database 
and to present tiie subscriber message in an audio form to a human operator; and 

a data entry conq)onent configured to obtain data firom the human operator derived 
firom the subscriber message. 
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